SimiHawk at SemEval-2016 Task 1: A Deep Ensemble System for Semantic Textual Similarity

This paper describes the SimiHawk system submission from UMass Lowell for the core Semantic Textual Similarity task at SemEval2016. We built four systems: a small featurebased system that leverages word alignment and machine translation quality evaluation metrics, two end-to-end LSTM-based systems, and an ensemble system. The LSTMbased systems used either a simple LSTM architecture or a Tree-LSTM structure. We found that of the three base systems, the feature-based model obtained the best results, outperforming each LSTM-based model’s correlation by .06. Ultimately, the ensemble system was able to outperform the base systems substantially, obtaining a weighted Pearson correlation of 0.738, and placing 7th out of 115 participating systems. We find that the ensemble system’s success comes largely from its ability to form a consensus and eliminate complementary noise from its base systems’ predictions.